Learning by seeing - associative learning of visual features through mental simulation of observed action

Internal representations employed in cognitive tasks have to be embodied. The flexible use of such grounded models allows for higher-level function like planning ahead, cooperation and communication. But at the same time this flexibility presupposes that the utilized internal models are interrelating multiple modalities. In this article we present how an internal body model serving motor control tasks can be recruited for learning to recognize movements performed by another agent. We show that—as the movements are governed by an equal underlying internal model—it is sufficient to observe the other agent performing a series of movements and that there is no supervised learning necessary, i.e. the learning agent does not require access to the performing agents postural information (joint configurations). Instead, through the shared underlying dynamics the mapping can be bootstrapped by the observing agent from the sequence of visual input features.